Natural language understanding (NLU) has the goal of programming computers to simulate understanding of natural languages by computers in order for them to perform actions involving language including answering questions, finding specified kinds of facts in ordinary “unstructured” text, and carrying on a dialog with humans in a natural language. Broadly speaking there are two approaches to NLU, symbolic and statistical, although combinations of the two often prove best. The symbolic approach typically involves rule-based methods and a knowledge representation language (KRL) such as lambda calculus or predicate calculus. Symbolic NLU systems analyze the meaning of English expressions in the process of constructing expressions in their KRL. These KRL expressions represent the meaning of the input English in a manner that is typically less ambiguous than the input English and in a manner that can be used subsequently by computers for inference and analysis. Natural languages such as English are inherently ambiguous in many ways. Much of the challenge in devising NLU systems involves finding ways to appropriately and efficiently reduce or eliminate such ambiguity as the data structures of the KRL are constructed.
The ultimate goal of NLU is to enable computer programs to understand and communicate using spoken and written language at least as well as adult native speakers of a natural language communicate with each other. The current state of the art is far from reaching this goal, although current technology does a good job for restricted or specialized applications, such as answering questions on a predetermined set of topics, finding documents or sentences that are related to a given phrase or to key words, or interacting with a human user in a restricted manner to make airline reservations. Even early systems could appear to understand language far beyond their actual capabilities because users tend to see human-like intelligence where it does not exist. For example an early NL program imitated a psychologist and asked questions such as “How does this make you feel?”. This system simply matched parts of the input statements from the user to a prefabricated list of responses and used a few stock phrases, such as “Tell me more about that” to fall back on when no matches were found.
When computer programs can: (1) carry on a sustained dialog, (2) keep track of the current topic and changes in that topic, (3) initiate and respond to communication goals appropriately, (4) make appropriate inferences during the dialog, (5) handle the many non-literal uses of language such as metaphor, (5) understand what the other parties in a dialog probably understand or need to know, and (6) guess appropriately about what the communication goals of the other dialog participants may be as the dialog progresses these may be indications that we are approaching the achievement of NLU's ultimate goal. All of these aspects of natural language communication are being investigated in the academic community. It has been argued that NLU is “Artificial Intelligence complete,” meaning that most of the goals of the broader field of Artificial Intelligence will need to be solved to reach this overall NLU goal.
Many of these performance criteria can currently be achieved for specialized applications, where the goals of the dialog and the language itself are limited. The key indicator that NLU systems have achieved something comparable to human-level competence may be their capability to perform the listed actions on an unlimited variety of subjects as human beings can. Of course, progress in any area has the potential to enable new commercial applications. Also, progress in increasing the generality of NLU systems means that fewer new distinct commercial products must be developed (or developed from scratch) to address new applications. The value of successfully developing more broadly capable NLU systems easily justifies the cost of gathering the additional knowledge needed to support their operation. The key is to demonstrate that a less specialized system can support new applications once its vocabulary and knowledge are large enough. Similar arguments have been made for the even more ambitious goal of creating a general artificial intelligence.
In about the last two decades much NLU research has emphasized statistical analysis primarily to minimize or to avoid applying large sets of hand crafted rules about language structure and use, which are labor intensive to collect. Unlike symbolic or rule-based approaches, these statistical approaches seek to avoid depending on large manually or semi-manually acquired collections of general common sense knowledge, domain specific knowledge and detailed specifications of grammar. Limitations of symbolic approaches that are often cited by advocates of statistical approaches include the changing and expanding nature of proper grammar and vocabulary as well as the labor intensive nature of specifying the kind of complex declarative knowledge just described.
Statistical approaches have their own limitations. Some statistical methods attempt to train the NL system based on a “training set” of hand annotated example problem-solution sets. These annotations can range from syntactic categories of words and phrases to a variety of “tags” that indicate semantic information. In contrast “unsupervised” training techniques require the system to achieve the desired performance by learning from example problem-solution pairs without any annotations being provided. The field concerned with ways to train computer programs in these ways is “machine learning”.
Some applications of NL, such as translating from one natural language to another, can achieve fairly high performance (at least on a first pass prior to review by expert human translators) without constructing any expressions in a knowledge representation language. Consequently, statistical techniques are appropriate for many translation tasks. Other applications such as question answering benefit significantly from the use of a KRL. The KRL can reflect details of NL such as how statements are quantified and whether or not they are negated. Such linguistic features are key for question answering. Machine leaning has difficulty coping with such features in the absence of a KRL. Recently at least one research group has begun to combine machine learning with the construction of a KRL by using problem-solution pairs that involve a small number of hand-crafted templates of expressions in a KRL that have associated parameters. When values of the parameters for their templates are determined, the resulting expressions resemble predicate calculus. For their system the machine learning task is to map expressions in natural language to these templates with the correct values of the parameters substituted, resulting in a fully specified expression in their KRL. One of their central challenges was to design the templates so that existing machine learning algorithms would be effective.
In view of the foregoing, it is desired to provide systems and methods for improving the efficiency of syntactic and semantic analysis in automated processes for NLU.